1. Field of the Invention
This invention relates generally to computer network communications and, more particularly, to cache techniques for network documents.
2. Description of the Related Art
When a computer network user begins a communication session over the Internet, the user can request data files from an Internet-connected computer called an origin server using the hypertext transfer protocol (HTTP). An origin server is typically a web server on which a given resource resides or is to be generated. These data files are typically written or defined in a type of programming code or mark-up language called hypertext mark-up language (HTML), and can be viewed or displayed through a graphical user interface (GUI) program, such as “Netscape Communicator” from Netscape Communications Corporation or “Internet Explorer” from Microsoft Corporation. Such data files may specify a mixture of content including: text, graphics (image), audio, and video. The network nodes and collection of HTML are commonly referred to as the “World Wide Web” (“WWW”), and the GUI program to view the files is called a (“web browser”). A collection of related data files under a common Internet network domain name is commonly referred to as a Web site.
A network user can request a Web page (or page) on a Web site by clicking on a link on a Web page in the browser window or by typing in a uniform resource locator (URL) address in the browser location toolbar. When a Web page from a Web site is requested, the browser takes charge of displaying the returned collection of data files as a complete Web page. For example, if a Web page includes text, animated images, and audio, the web browser will retrieve the Web page and display the text, show the images, and play the audio sounds. The browser request is sent from the user computer to a series of switches and routers toward the appropriate origin web server, where the requested Web page is identified and/or generated, and returned to the user for display.
Because of extremely heavy data traffic over the Internet, cache techniques are being used to improve the speed of returning requested Web pages to network users. One way is by having a proxy server. A proxy server is a device that has a data store, cache, or cache memory that typically contains local copies of frequently requested Web pages. Proxy servers are distributed around the Internet so as to be readily accessible by various servers. Thus, if a user requests a Web page, for example, the proxy server checks the local data store to determine if it has the corresponding page in cache for the URL request just received. If the proxy server identifies that it has the Web page for the URL requested, it will return the cached Web page, found in the local proxy server, to the requesting user, rather than relaying the request all the way back to the origin server. This reduces the time for the Web pages to be returned to the user.
Often, Web pages are dynamically generated at the time they are requested to allow for content flexibility. (Dynamically created pages are hereafter referred to as dynamic pages or dynamic Web pages.) In conjunction with a web server, a script or program usually generates such dynamic pages. Typically, query parameters are also passed to the page-generating script, so that the data drawn from the data source is limited based on, for example, user's input or selection. (The process of generating Web pages is herein referred to as page generation.) One skilled in the art will recognize that Web pages may be generated via other programmed means, for example, generating Web pages using a high-level programming language, such as C++, without utilizing a web server. (Static pages are those pages that are not dynamically generated.)
Conventionally, a dynamic page is generated in response to a user request. Thus, if ten users request a dynamic page containing identical information, the origin server has to generate such page ten times to satisfy those ten requests even if no piece of information changed from the first request to the tenth request. For example, although a Web page containing a list of bids for an auction item does not change very often (probably, once every few hours during the initial posting of the item), it is likely implemented as a dynamic page and generated anew upon each user request. This unnecessary generation of Web pages, thus, adds a substantial load on the origin server's resources. Hence, a way to generate pages only when the Web page content is no longer valid and a way to alleviate the resource requirements on the origin server are highly desired.
Dynamic pages are typically not cached. Dynamic Web pages typically change frequently, often minute by minute. For example, dynamic Web pages may be used to communicate the latest bid information at an Internet auction site, or flight arrival information at a travel site, or may be used to post remaining inventory at a retail-shopping site. Because dynamic pages are not cached, all user requests for dynamic pages must travel all the way back to the origin server at the source Web site and, in addition, are generated by the origin server. This places a high demand on the origin servers for handling traffic and for generating dynamic Web pages, and can slow down the response time for web requests.
One way of caching dynamic pages is by having such pages be refreshed (i.e., replaced with a fresher or newer page) every few minutes, e.g., all cached dynamic pages that are fifteen minutes old or older are refreshed. This method, however, means that dynamic pages that are still valid (i.e., contain up-to-date information) are unnecessarily refreshed thereby needlessly taxing the resources of the system. Furthermore, users may also receive invalid web pages during the time frame when such pages have not been refreshed and contain invalid data.
Another way of caching dynamic pages enables developers to split up a dynamic page to different portions based on whether it contains static information, such as information that does not need to change, or whether it contains dynamic content, such as information drawn from a database. The system then composes a complete web page whenever a request is received. This method, however, involves redundant requests for dynamic contents considering that dynamic contents from previous requests may still be valid, as well as unnecessary delay considering that the request for dynamic content has to be sent all the way back to the origin server.
If dynamic pages were cached using conventional technology, the pages kept in cache could quickly become out of date, being superceded by more recent information from the origin server. Alternatively, if cached dynamic pages were refreshed with sufficient frequency to ensure they remain valid, the rate of refresh would likely be such that much of the cache operation would be devoted to handling refresh operations. This would tax the resources of the cache and reduce the effectiveness of the cache storage.
FIG. 1 shows a conventional arrangement 100 for providing dynamic pages to Internet users. A user at a computer with a browser 102 communicates with the Internet 104 using a network connection 105. The browser sends over the Internet Web page requests that are received by an origin server 106. Requests for dynamic pages are relayed to an origin dynamic content server 108, which generates the Web pages from a data store 110. The requested dynamic pages are then sent back to the users and displayed in the users' browsers 102.
From the discussion above, it is apparent that there is a need for an Internet cache technique that can store dynamic pages thereby reducing the workload on web servers, but efficiently maintaining valid pages without unduly taxing cache resources. The present invention fulfills this need.